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Congressional  Committees 

In  response  to  the  Federal  Acquisition  Streamlining  Act  (fasa)  of  1994,  we 
reviewed  the  National  Aeronautics  and  Space  Administration’s  (nasa) 
Acquisition  Internet  Service  (nais).1  Specifically,  we  determined  whether 
nais  was  an  effective  mechanism  for  disseminating  procurement 
information  to  industry,  including  small  businesses.  In  addition,  we 
identified  the  status  of  efforts  to  develop  a  govemmentwide  electronic 
procurement  information  system  similar  to  nais. 

In  1993,  NASA  began  an  initiative  called  the  Midrange  Pilot  Program  for 
procurements  between  $25,000  and  $500, 000.2  The  purpose  of  the  program 
was  to  test  new  processes  and  tools  to  streamline  the  acquisition  process 
while  maintaining  or  improving  the  quality  of  procurements.  A  key 
objective  of  the  program  was  to  develop  and  test  an  alternative  to  paper 
documents  as  the  primary  means  of  communicating  procurement 
information  to  prospective  contractors.  After  researching  available 
options,  nasa  chose  to  implement  an  Internet-based  electronic 
procurement  notice  and  publication  system,  which  ultimately  became  nais. 

In  December  1996,  the  Department  of  Commerce,  in  partnership  with  the 
Government  Printing  Office,  introduced  CBDNet,  an  Internet  version  of 
the  Commerce  Business  Daily  (cbd),3  which  is  available  to  the  public  free 
of  charge.  Many  other  agencies  have  developed  a  variety  of  other 
electronic  posting  systems  to  help  improve  access  to,  and  the  visibility  of, 
their  contracting  opportunities. 


Results  in  Brief 


nais  is  a  simple,  effective,  and  user-friendly  system  for  disseminating 
information  on  contract  opportunities,  nais  has  contributed  to  the 
development  of  a  more  standardized  and  streamlined  acquisition  process 
at  nasa  and  provides  a  central  electronic  source  of  procurement 
information  from  nasa’s  decentralized  facilities.  It  allows  businesses  to 
obtain  procurement  information  immediately,  without  waiting  for  mail 


’P.L.  103-355,  section  5062,  authorized  NASA  to  conduct  a  test  of  electronic  notice  and  publication  for 
procurements  and  requires  GAO  to  report  on  the  results  of  NASA’s  test 

-In  fiscal  year  1997,  NASA  increased  the  threshold  to  $2  million.  NASA  officials  estimate  that  80  to 
90  percent  of  all  NASA’s  annual  contract  awards  are  in  the  midrange  category. 

:VThe  CBD  gives  a  brief  description  (or  synopsis)  of  federal  acquisitions  over  $25,000.  The  primary 
purposes  of  CBD  notices  are  to  improve  small  businesses’  access  to  acquisition  information  and 
enhance  competition  by  identifying  contracting  and  subcontracting  opportunities. 
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delivery  of  printed  information.  Vendors  especially  like  the  electronic  mail 
(e-mail)  notification  service  that  automatically  sends  announcements 
about  procurements  of  interest  to  them.  Vendor  feedback  about  nais  came 
primarily  from  small  businesses  and  was  generally  positive.  Procurement 
data  showed  that  offers  and  awards  to  small  businesses  did  not  change 
significantly  after  nais  implementation,  nasa  noted  that  data  limitations 
made  it  difficult  to  quantify  other  nais  benefits. 

NASA,  the  General  Services  Administration  (gsa),  and  other  federal 
agencies  are  working  together  to  develop  a  single,  govemmentwide 
Internet  entry  point  for  information  on  federal  procurement  opportunities. 
But  a  number  of  steps  must  still  be  taken  and  many  obstacles  remain. 

Even  if  the  new  system  is  successfully  developed  and  implemented 
govemmentwide,  current  statutory  requirements  for  publication  of 
procurement  notices  and  minimum  waiting  periods  for  mail  delivery  may 
continue  to  limit  the  potential  benefits  of  an  electronic  procurement 
information  system.  The  same  legislation  that  encouraged  nasa  and  others 
to  work  together  also  requires  the  Office  of  Federal  Procurement  Policy  to 
submit  to  Congress  annual  reports  assessing  compliance  with  the 
requirement  to  provide  direct  access  to  procurement  information  through 
a  single  govemmentwide  electronic  entry  point. 


Background 


In  the  early  1990s,  federal  agencies  and  Congress  recognized  that  doing 
business  electronically  could  save  resources,  streamline  the  procurement 
process,  and  improve  the  private  sector’s  access  to  federal  contracting 
opportunities,  nasa  first  used  nais  at  its  Marshall  Space  Flight  Center  in 
1994  to  post  midrange  procurement  opportunities  on  the  Internet,  nais 
services  and  features  expanded  rapidly,  and  the  system  began  to  be  used 
NASA-wide  in  1995.  It  was  the  first  agencywide  procurement  information 
system  on  the  Internet. 

With  some  exceptions,  federal  agencies  must  publish  notices  of 
competitive  contracting  opportunities  above  $25,000  in  the  cbd.  To 
accommodate  timely  delivery  of  notices  through  a  paper-based  mail 
delivery  system,  the  law  generally  prescribes  a  15-day  minimum  interval 
between  publication  of  a  notice  in  the  cbd  and  the  issuance  of  a 
solicitation  by  an  agency.  After  a  solicitation  is  issued,  agencies  generally 
must  give  contractors  a  minimum  of  30  days  to  prepare  and  deliver  bids  or 
offers.  In  support  of  nasa’s  test  of  alternatives  to  the  use  of  paper 
documents  for  communicating  business  opportunities  to  prospective 
contractors,  fasa  authorized  nasa  to  waive  the  cbd  notice  and  waiting 
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period  requirements  for  4  years  (until  Oct.  13, 1998)  and  for  up  to 
$100  million  in  procurement  life-cycle  costs. 

Congress  has  supported  the  use  of  federal  electronic  commerce4  and 
electronic  access  to  procurement  information.  Under  fasa,  this  capability 
was  to  be  provided  through  the  Federal  Acquisition  Computer  Network 
(facnet),  a  govemmentwide  systems  architecture  for  acquisitions  based 
on  electronic  data  interchange,  which  is  the  computer-to-computer 
exchange  of  routine  business  documents  using  standardized  data  formats. 
facnet  implementation  proved  to  be  problematic,  however,  and  in  1997, 
section  850  of  the  National  Defense  Authorization  Act  for  Fiscal  Year  1998 
replaced  the  preference  for  facnet  with  a  more  flexible  policy  that 
promotes  the  use  of  electronic  commerce  techniques  whenever 
practicable  or  cost-effective.5 

Section  850  also  requires  that  in  implementing  electronic  commerce, 
agencies  apply  standards  consistent  with  nationally  and  internationally 
recognized  standards  that  broaden  interoperability  and  ease  of  use. 
Agency  notices  of  procurement  opportunities  must  be  provided  in  a  form 
that  allows  convenient  and  easy  user  access  through  a  single 
govemmentwide  point  of  entry.  While  CBDNet  satisfies  the  requirement 
with  respect  to  notices,  neither  it  nor  other  currently  available  electronic 
procurement  posting  systems  satisfy  the  requirement  for  a  single 
govemmentwide  point  of  entry  for  access  to  solicitations  and  other 
procurement  information. 

The  law  also  requires  that  the  Administrator  of  the  Office  of  Federal 
Procurement  Policy  (ofpp)  submit  to  Congress  yearly  reports  through  2003 
on  the  progress  being  made  in  implementing  section  850.  The  reports  must 
include  a  strategic  plan  for  implementation  of  govemmentwide  electronic 
commerce  and  an  assessment  of  compliance  with  the  requirement  for  a 
single  point  of  entry. 


■‘Electronic  commerce  is  conducted  using  electronic  techniques  for  business  transactions,  including 
electronic  mail  or  messaging,  World  Wide  Web  technology,  electronic  bulletin  boards,  purchase  cards, 
electronic  funds  transfers,  and  electronic  data  interchange. 

5 We  reported  previously  on  numerous  concerns  about  FACNET  implementation,  including  FASA’s 
requirements.  See  Acquisition  Reform:  Obstacles  to  Implementing  the  Federal  Acquisition  Computer 
Network  (GAO/NSIAD-97-26,  Jan.  3, 1997)  and  Acquisition  Reform:  Classes  of  Contracts  Not  Suitable 
for  the  Federal  Acquisition  Computer  Network  (GAO/NSIAD-97-232,  Sept.  17,  1997). 
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Since  1994,  nais  has  evolved  into  a  procurement  information  system  with 
many  operational  features  that  are  effective,  easy  to  use,  and  beneficial  to 
nasa  and  vendors.  By  eliminating  many  of  the  steps  in  the  paper-based 
process  of  publicizing  contract  opportunities  and  issuing  solicitations,  nais 
has  contributed  to  a  more  streamlined  agency  acquisition  process. 
Feedback  from  nais  users,  particularly  small  businesses,  has  generally 
been  positive.  However,  the  lack  of  data  makes  it  very  difficult  to  quantify 
specific  benefits  of  nais,  such  as  the  amount  of  time  it  actually  saves,  nasa 
exceeded  the  $100~million  ceiling  authorized  by  fasa  for  the  test.  After  nais 
implementation,  the  number  of  offers  per  contract  and  small  businesses’ 
share  of  contract  actions  and  dollars  showed  some  increases. 


Since  July  1994,  nais  has  evolved  to  provide  a  broad  range  of 
procurement-related  functions  and  information.  For  example,  in 
November  1995,  NASA  added  an  on-line  procurement  reference  library.  In 
March  1997,  nasa  completed  the  nais  Electronic  Posting  System  (eps), 
which  enables  procurement  staff  anywhere  in  the  agency  to  prepare  and 
post  notices  and  place  solicitation  files  directly  onto  the  nais  Internet  site. 
In  March  1997,  nasa  also  added  an  e-mail  notification  service  that 
automatically  notifies  subscribers  of  new  procurement  information 
available  on  nais. 

We  found  that  the  system’s  operational  features  were  easy  to  use  and 
worked  as  described.  Procurement  staff  demonstrated  how  they  use  the 
EPS  to  generate,  edit,  and  post  notices  and  place  solicitation  files  directly 
onto  the  Internet.  The  EPS  automatically  formats  each  nais  posting,  inserts 
links  to  other  relevant  documents  and  external  information,  updates  the 
business  opportunities  index  library,  and  constructs  and  feeds  messages  to 
the  nais  e-mail  notification  service.  Appendix  I  provides  more  information 
on  the  development  and  evolution  of  the  eps  and  the  approach  nasa  used 
to  design,  develop,  and  operate  the  system. 

By  going  to  the  nais  Internet  site,  users  can 

•  identify,  at  one  location,  notices  of  and  solicitations  for  competitive 
business  opportunities  and  awards  over  $25,000  at  all  nasa  centers; 

•  search  for  business  opportunities  by  type  of  product  and/or  service,  nasa 
center,  or  posting  date; 

•  locate  and  download  documents  related  to  a  specific  procurement;  and 


NAIS  Is  User-Friendly  and 
Works 


NAIS  Is  an  Effective 
Information  Service 
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•  view  summaries  of  NASA’s  current  contracts  (with  contractor  name,  value, 
obligations,  and  description)  and  search  current  contracts  by  state, 
congressional  district,  business  type,  or  type  of  product  and/or  service. 

The  e-mail  notification  service  automatically  transmits  announcements  to 
subscribers  immediately  after  new  procurement  information  is  posted  on 
nais.  This  allows  subscribers  to  track  developments  in  the  solicitation 
cycle  and  learn  of  new  releases.  Subscribers  can  register  free  of  charge  by 
providing  an  e-mail  address  and  selecting  the  types  of  contracting 
opportunities  in  which  they  have  the  greatest  interest.  Thus,  instead  of 
having  to  search  for  information,  subscribers  can  have  it  selected  and 
delivered  to  their  desktop  electronically.  As  of  July  1998,  there  were  over 
7,100  subscribers  to  the  e-mail  notification  service.  According  to  NASA,  user 
feedback  showed  that  this  service  was  the  most  popular  feature  of  nais. 


NAIS  Contributes  to  a 
Streamlined  and 
Accessible  Procurement 
System 


One  of  nasa’s  goals  in  establishing  nais  was  to  streamline  and  standardize 
the  agency’s  procurement  process  by  using  the  Internet  as  the  gateway  to 
contracting  opportunities  for  businesses,  nais  streamlined  or  eliminated 
many  of  the  steps  required  by  the  paper-based  process  for  publicizing 
synopses  of  contracting  opportunities  and  issuing  solicitations. 

The  nais  process  involves  fewer  steps  for  procurement  staff  than  the  paper 
process.  The  eps,  for  example,  automatically  formats  notices,  performs 
edit  checks,  automatically  posts  notices  on  the  Internet  through  the  nais 
server,  and  transmits  them  to  the  Government  Printing  Office  for 
publication  in  the  cbd.  Using  the  paper  process,  procurement  staff  have  to 
perform  analogous  functions  manually  and  must  also  respond  to  requests 
for  copies  of  solicitations  by  preparing  mailing  labels  and  mailing  the 
copies.  With  nais,  the  solicitation  files  are  available  for  review  and 
retrieval  (“downloading”)  through  the  Internet. 

We  believe  the  nais  process  is  also  easier  for  contractors  to  use.  Using  the 
paper-based  system,  contractors  must  either  scan  the  cbd  or  periodically 
call  or  visit  each  of  NASA’s  procurement  offices  to  identify  contracting 
opportunities.  In  contrast,  using  nais,  contractors  can  search  for  and  view 
as  many  synopses  and  solicitations  as  they  wish  on-line.  They  can  search 
by  specific  commodity,  service,  or  location;  view  and  provide  comments 
on  draft  solicitations;  and  access  the  Federal  Acquisition  Regulation,  nasa 
procurement  policies  and  procedures,  and  nasa  procurement  forms. 
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User  Feedback  Was 
Positive 


Lack  of  Data  Makes  It 
Difficult  to  Quantify  NAIS 
Benefits 


nasa  officials  point  to  the  absence  of  protests6  from  contractors 
concerning  nais  and  the  lack  of  complaints  about  najs  to  the  nasa 
Ombudsman  Program7  as  evidence  that  the  contractor  community 
approves  of  nais. 


Both  businesses  and  nasa  procurement  staff  have  provided  positive 
feedback  about  nais.  Overall,  industry  responses  to  specific  nais 
operational  features  were  positive.  In  a  tally  of  on-line  comments  from  nais 
users,  87  percent  of  respondents  said  they  found  nais  to  be  beneficial.  The 
respondents  identified  themselves  primarily  as  small,  small  disadvantaged, 
or  women-owned  businesses.  For  the  most  part,  feedback  from  the 
procurement  staff  was  positive,  and  comments  were  often  similar  to  those 
made  by  businesses.  Federal  electronic  commerce  officials  from  other 
agencies  and  industry  representatives  told  us  that  systems  with  the 
operational  features  of  nais  can  benefit  both  industry  and  government, 
especially  when  all  buying  organizations  are  using  the  same  system. 

An  on-line  nais  feedback  system  was  established  in  1995  to  obtain  input, 
especially  from  businesses,  about  the  system.  The  nais  team8  also  requests 
feedback  from  the  procurement  staff  at  nasa  centers.  This  feedback  was 
compiled  in  December  1997  and  again  in  June  1998.  The  team  compiles 
survey  feedback  data,  develops  a  consolidated  summary  of  industry 
suggestions,  and  uses  the  summary  to  determine  which  changes  and 
enhancements  should  be  made. 


nasa’s  evaluation  plan  for  the  Midrange  Pilot  Program  did  not  include  a 
separate  nais  evaluation  plan.  In  May  1998,  nasa  published  the  nais 
Business  Case,  which  provides  details  of  current  nais  features,  along  with 
its  history,  benefits,  and  funding  alternatives.  However,  nasa  did  not 
collect  data  on  or  evaluate  individual  procurements  advertised  through 
nais.  The  information  needed  to  determine  how  much  time  nais  actually 
saved  or  to  quantify  other  nais  benefits,  such  as  cost  avoidance  for  the 
procurement  process,  was  not  available.  We  did  not  find  any  evidence  that 


(iA  protest  is  a  written  objection  by  an  interested  party  to  a  solicitation  or  proposed  award  or  award  of 
a  contract  Protests  are  usually  filed  either  with  the  procuring  agency  or  with  GAO. 

7The  program  was  established  in  October  1995  to  facilitate  communications  between  NASA  and 
current  and  potential  contractors.  It  provides  offerors,  potential  offerors,  and  contractors  with  a  single 
point  of  contact  to  address  their  concerns. 

^he  NAIS  team  consists  of  a  small  group  of  procurement  and,  in  some  cases,  technical  staff  from  each 
NASA  center  and  headquarters. 
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nais  had  negative  effects,  and  there  is  substantial  qualitative  evidence  of 
the  positive  impact  of  nais.  Appendix  II  provides  more  information  on  our 
assessment  of  the  nais  Business  Case. 


NASA  Exceeded  Waiver  Although  FASA  allowed  NASA  a  4-year  test  authority  to  waive  cbd 

Authority’s  Dollar  Ceiling  publication  requirements  and  waiting  periods  up  to  a  limit  of  $100  million 

in  life-cycle  costs,  nasa  did  not  establish  a  mechanism  to  track  compliance 
with  the  waiver  provision,  nasa  officials  told  us  that  when  the  waiver 
authority  was  originally  requested,  they  anticipated  that  nais,  though  not 
yet  developed,  would  be  used  primarily  by  the  Marshall  Space  Flight 
Center,  the  focal  point  of  the  pilot  test,  and  that  the  $  100-million  ceiling 
would  be  adequate  to  cover  Marshall’s  procurements.  However,  the 
Internet  posting  capability  was  rolled  out  to  all  nasa  centers  by  April  1995 
and  was  being  used  agencywide  for  all  competitive  procurements  over 
$25,000  1  month  before  nasa  began  waiving  the  cbd  notice  requirements  in 
October  1995. 

In  early  1997,  nasa  officials  realized  that  the  $  100-million  ceiling  authorized 
by  fasa  for  the  test  would  not  cover  agencywide  use  of  nais.  In  May  1997, 
the  Administration  submitted  a  legislative  proposal  to  increase  the  ceiling 
for  nasa’s  use  of  the  waiver  authority  and  extend  the  test  period  from  4  to 
6  years.  Congress  did  not  act  on  the  proposal.  NASA  used  the  cbd  waiver 
authority  through  the  end  of  the  fiscal  year-September  30, 1997.  nasa 
calculated  that  it  awarded  approximately  1,585  contracts  worth  a  total  of 
about  $177  million  using  the  cbd  waiver  authority. 


No  Significant  Changes  in 
Offers  and  Awards 
Reported  After  NAIS 
Implementation 


In  an  effort  to  determine  the  impact  of  nais  on  the  amount  of  competition 
for  awards,  nasa  compared  the  number  of  vendors’  proposals  received 
before  and  after  nais  implementation,  nasa  compared  the  number  of  offers 
received  per  contract  action  in  fiscal  year  1994-the  year  prior  to 
agencywide  use  of  NAis-to  the  number  of  offers  per  contract  action  in 
fiscal  year  1997-a  year  in  which  all  midrange  procurements  were  only 
advertised  through  nais.  As  table  1  shows,  the  average  number  of  offers 
received  per  acquisition  increased  from  6.1  to  7.2  after  nais 
implementation,  nasa  said  in  the  nais  Business  Case  that  the  increase  in 
the  average  number  of  offers  received  per  acquisition  was  especially 
promising  because  the  majority  of  these  acquisitions  were  set  aside  for 
small  businesses. 
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Table  1 :  NAIS’s  Impact  on  Competition 


Dollars  in  thousands 

Fiscal  year 

Procurement  activities3 

1994 

1997 

Number  of  contracts  awarded 

644 

396 

Offers  received  from  businesses 

3,959 

2,852 

Offers  per  contract 

6.1 

7.2 

Number  of  contracts  to  small  businesses 

380 

252 

Small-business  share  (percent) 

59.01 

63.64 

Dollar  value  of  contract  awards 

$3,296 

$930 

Dollars  awarded  to  small  businesses 

$431 

$283 

Small-business  dollar  share  (percent) 

13.1 

30.4 

Competitive  contract  actions  over  $25,000. 


We  also  calculated  and  compared  the  percentages  of  contract  actions  and 
dollars  awarded  to  small  businesses  in  fiscal  years  1994  and  1997, 
respectively.  Table  1  shows  these  calculations.  The  percentage  share  of 
awards  to  small  businesses  did  not  change  significantly  after  nais 
implementation,  but  the  percentage  share  of  dollars  awarded  to  small 
businesses  increased  substantially  in  1997.  A  NASA  official  said  this  was 
due  in  large  part  to  a  drop  in  awards  to  large  businesses  and  not  to  any 
substantial  change  in  dollars  awarded  to  small  businesses. 


Efforts  Are  Underway 
to  Establish  a 
Govemmentwide  EPS 


Today  most  federal  agencies  use  the  Internet  in  some  way  to  advertise 
their  contracting  opportunities.  Vendors  search  all  these  various  Internet 
sites  for  government  procurement  information.  Recent  legislation  requires 
the  government  to  provide  direct  access  to  notices  of  agencies’ 
requirements  and  solicitations  through  a  single  govemmentwide  electronic 
point  of  entry,  nasa,  gsa,  and  other  agencies  are  testing  a  multiagency 
electronic  posting  system.  Agency  officials  are  concerned  that  the  benefits 
of  providing  direct  electronic  access  to  contracting  opportunities  may  be 
offset  by  existing  statutory  requirements  for  publication  of  procurement 
notices  and  minimum  waiting  periods. 


Status  of  Efforts  to 
Develop  a 

Govemmentwide  EPS 


According  to  the  San  Antonio  Electronic  Commerce  Resource  Center,9 
more  than  400  different  Internet  sites  provide  federal  procurement 
information.  This  multitude  of  sites  makes  it  difficult  and  time-consuming 


Electronic  Commerce  Resource  Centers  are  a  network  of  government-sponsored  centers  that  provide 
electronic  commerce  training  and  support  to  the  contractor  community. 


Page  8 


GAO/NS IAD-99-3 7  Acquisition  Reform 


B-280239 


for  contractors  to  obtain  govemmentwide  information  on  contracting 
opportunities. 

The  March  1998  report,  Electronic  Commerce  for  Buyers  and  Sellers,  A 
Strategic  Plan  for  Electronic  Federal  Purchasing  and  Payment,10  described 
planned  enhancements  to  the  electronic  version  of  the  cbd,  CBDNet, 
which  now  provides  a  single  point  of  entry  only  for  procurement  notices. 
gsa  and  nasa  approached  the  Department  of  Commerce  and  the 
Government  Printing  Office  to  form  an  interagency  team  to  implement  an 
enhanced  CBDNet  and  to  make  an  Internet-based  EPS  with  features 
comparable  to  those  of  the  nais  system  available  to  the  entire  federal 
sector.11  However,  the  agencies  could  not  reach  agreement  on  an 
enhanced  CBDNet  implementation  approach.  Among  unresolved  issues 
were  the  source  and  level  of  funding  needed  for  the  project,  the  costs  to 
develop  and  implement  the  system,  and  the  potential  financial  impact  on 
the  operating  and  user  agencies. 

nasa,  gsa,  and  other  members  of  the  Interagency  Acquisition  Internet 
Council12  are  developing  and  testing  a  pilot  multiagency  posting  system 
derived  from  the  nais  eps  to  enable  agencies  to  post  notices,  solicitations, 
and  other  acquisition-related  documentation  directly  onto  the  Internet, 
where  it  would  be  available  through  a  single  point  of  entry,  gsa  is  leading 
this  effort  and  spent  over  $400,000  in  the  past  year  developing  this 
multiagency  EPS  software.  In  October  1998,  gsa  began  using  the  pilot 
software  as  the  agency’s  single  eps.  If  testing  demonstrates  that  the 
multiagency  eps  is  capable  of  providing  effective  access  to  notices  and 
solicitations  through  a  single  point  of  entry,  the  Administrator  of  the  Office 
of  Federal  Procurement  Policy  will  consider  designating  it  as  the  single 
govemmentwide  point  of  entry  required  by  section  850. 


U)Prepared  by  the  Electronic  Processes  Initiatives  Committee  of  the  President’s  Management  Council 
and  submitted  to  Congress  in  response  to  the  requirement  in  section  850  of  the  Fiscal  Year  1998 
Department  of  Defense  Authorization  Act  for  a  strategic  plan  for  the  implementation  of  a 
govemmentwide  electronic  commerce  capability. 

uThe  Department  of  Commerce  is  responsible  for  keeping  the  business  community  informed  about 
federal  procurement  actions  through  publication  of  the  CBD.  In  1961,  the  Government  Printing  Office 
assumed  CBD  publication  and  distribution  functions.  However,  all  editorial  functions  remained  within 
the  jurisdiction  of  the  Department  of  Commerce. 

12The  Council  was  established  in  May  1996  to  seek  and  promote  ways  to  optimize  the  use  of  the 
Internet  in  streamlining  the  federal  acquisition  process  and  increasing  communication  of  federal 
acquisition-related  information. 
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Challenges  to 

Govemmentwide 

Implementation 


Conclusions 


The  difficulty  of  establishing  a  single  govemmentwide  electronic  point  of 
entry  for  federal  procurement  opportunities  was  emphasized  by  several 
agency  officials,  who  said  they  were  concerned  about  the  costs  involved  in 
developing,  operating,  and  maintaining  such  a  system;  cost  sharing  among 
different  agencies;  and  the  linkages  that  need  to  be  established  with 
existing  agency  systems  through  the  Internet.  They  also  questioned  how 
and  when  decisions  would  be  made  concerning  who  would  lead  such  a 
project,  who  would  be  responsible  for  operating  and  maintaining  the 
system,  and  who  would  mandate  its  use  by  agencies. 

Agency  officials  also  said  that  the  benefits  of  a  govemmentwide  electronic 
notice  capability  could  be  enhanced  through  changes  to  the  current 
statutory  minimum  waiting  periods  for  notices  and  solicitations,  which 
were  fashioned  around  a  paper-based  process.  Acquisition  officials  noted, 
in  particular,  that  the  law  requires  a  15-day  waiting  period  after 
publication  of  a  notice  before  a  solicitation  can  be  issued,  even  though  the 
Internet  enables  agencies  to  publicize  notices  and  provide  solicitation  files 
simultaneously,  giving  contractors  immediate  access  to  these  documents 
and  saving  time  in  the  procurement  process.  Agency  officials  said  that 
changing  these  requirements  could  increase  the  benefits  of  using 
electronic  commerce,  ofpp  plans  to  study  what  type  of  legislative  action 
may  be  needed  to  enhance  the  benefits  of  electronic  publication  of 
business  opportunities. 


NASA  has  shown  that  it  can  successfully  use  nais  to  effectively  disseminate 
solicitations  and  other  procurement  information,  giving  contractors  easy 
and  immediate  access  to  business  opportunities  at  each  of  its 
geographically  dispersed  procurement  offices.  Feedback  from  users  has 
been  positive,  and  the  system  has  allowed  increased  standardization  of  the 
procurement  process  agencywide.  However,  benefits  such  as  the  time 
savings  obtained  through  use  of  the  system  cannot  be  readily  quantified 
because  of  data  limitations.  While  nasa  believes  that  nais  has  had  a 
positive  impact  on  competition,  this  conclusion  was  based  on  only  2  years 
of  data. 

nasa’s  on-line  electronic  posting  system  was  the  model  for  the  pilot 
multiagency  eps  software  that  is  being  tested  by  gsa,  nasa,  and  several 
other  agencies.  If  successful,  this  eps  may  be  considered  for 
govemmentwide  expansion  and  designation  as  the  single  govemmentwide 
point  of  entry  for  both  notices  and  solicitations.  While  the  March  1998 
report  assessing  electronic  commerce  initiatives  discussed  enhancements 
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to  the  CBDNet,  it  did  not  discuss  the  multiagency  EPS  pilot  program.  If  a 
govemmentwide  EPS  is  successfully  implemented,  its  benefits  potentially 
could  be  enhanced  by  changing  the  statutory  notice  and  waiting  periods. 


Recommendations 


We  recommend  that  the  Administrator  of  the  Office  of  Federal 
Procurement  Policy  describe  the  current  plan  for  implementation  of  a 
govemmentwide  single  point  of  entry  to  procurement  information  in  its 
next  annual  report  to  Congress.  This  plan  should  take  into  consideration 
the  status  of  and  lessons  learned  from  the  multiagency  EPS  pilot  and  its 
potential  as  the  basis  for  a  comprehensive  govemmentwide  electronic 
posting  system. 

We  also  recommend  that  the  Administrator  of  the  Office  of  Federal 
Procurement  Policy  determine  whether  the  issues  raised  by  agency 
officials  concerning  procurement  notice  requirements  set  forth  in  statute, 
including  waiting  periods,  require  legislative  action  and  make 
recommendations  to  Congress  on  legislative  changes,  if  necessary. 


Agency  Comments 


In  commenting  on  the  draft  of  this  report,  NASA  and  ofpp  generally  agreed 
with  our  findings  and  recommendations,  nasa  stated  that  it  was  proud  of 
the  nais  staff  for  pioneering  the  use  of  the  Internet  in  federal  procurement; 
and  our  review  largely  corroborated  the  agency’s  assessment  of  the  value 
Of  NAIS. 

ofpp  stated  that  nais  has  created  the  foundation  for  the  ongoing 
multiagency  pilot  of  an  electronic  posting  system.  If  successful,  the  EPS 
will  provide  easy  access  to  procurement  information  across  the 
government  through  a  single  point  of  entry,  ofpp  was  encouraged  by  the 
promising  findings  identified  in  the  report,  ofpp  also  commented  on  our 
recognition  that  current  statutory  requirements  for  procurement  notices 
and  minimum  waiting  periods  may  limit  the  potential  benefits  of  electronic 
commerce  and  indicated  that  the  feasibility  of  legislative  action  would  be 
studied. 

The  comments  from  nasa  and  ofpp  are  reprinted  in  their  entirety  in 
appendix  III  and  IV,  respectively. 
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Scope  and 
Methodology 


To  determine  whether  the  nais  electronic  notice  and  publication  system 
was  an  effective  mechanism  for  disseminating  procurement  information, 
we  obtained  information  about  the  system’s  design,  development, 
operation,  costs,  and  benefits.  We  obtained  documents  from  and 
interviewed  officials  at  nasa  Headquarters  in  Washington,  D.C.,  and  at  the 
Marshall  Space  Flight  Center  (the  lead  center  for  nais  project 
management)  in  Huntsville,  Alabama.  We  also  accessed  NASA’s 
procurement  Internet  site  (http://procurement.nasa.gov)  and  reviewed  the 
on-line  nais  operational  features  available  to  both  public  users  and  nasa 
contracting  staff  to  determine  whether  the  system  worked  as  nasa 
represented.  To  evaluate  nasa’s  claim  that  nais  usage  had  a  positive  impact 
on  competition  and  was  beneficial  to  small  businesses,  we  compared  nasa 
procurement  data  for  fiscal  year  1994  (the  year  prior  to  agencywide  use  of 
nais)  with  data  for  fiscal  year  1997.  We  did  not  independently  verify  the 
nasa  procurement  data  obtained  from  the  nasa  acquisition  management 
information  system. 

We  also  reviewed  nasa’s  evaluation  of  the  system  contained  in  the 
May  1998  nais  Business  Case.  We  reviewed  the  data  in  the  nais  Business 
Case,  including  calculations  of  cost  avoidance  and  other  benefits 
attributed  to  nais.  We  examined  on-line  industry  feedback  and  narrative 
responses  from  vendors  and  nasa  buyers  about  nais.  To  determine  the 
agency’s  compliance  with  statutory  requirements,  we  reviewed  the  goals, 
objectives,  and  time  frames  established  by  fasa  for  nasa’s  test  of  electronic 
notice  and  publication. 

To  assess  the  status  of  efforts  to  establish  a  govemmentwide  eps  based  on 
the  nais  model,  we  asked  electronic  commerce  coordinators  from  several 
federal  agencies,  federal  Electronic  Commerce  Resource  Centers,  and  two 
industry  groups  for  their  observations.  We  received  responses  from 
coordinators  at  five  agencies,  two  resource  centers,  and  one  industry 
group.  We  also  interviewed  acquisition  officials  at  gsa,  the  Department  of 
Commerce,  and  ofpp  to  obtain  information  on  interagency  efforts  to 
develop  a  govemmentwide  electronic  procurement  posting  system.  We 
also  reviewed  the  goals,  objectives,  and  milestones  established  for  the 
govemmentwide  electronic  commerce  program  by  the  National  Defense 
Authorization  Act  for  Fiscal  Year  1998. 

We  performed  our  work  between  January  and  December  1998  in 
accordance  with  generally  accepted  government  auditing  standards. 
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We  are  sending  copies  of  this  report  to  the  Administrator,  NASA;  the 
Administrator,  ofpp;  and  the  Director,  Office  of  Management  and  Budget. 
We  will  also  make  copies  available  to  others  upon  request. 

Please  contact  me  at  (202)  512-4841  if  you  or  your  staff  have  any  questions 
concerning  this  report.  Major  contributors  to  this  report  are  listed  in 
appendix  V. 


Allen  Li 

Associate  Director 
Defense  Acquisitions  Issues 
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Chairman 
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Chairman 

The  Honorable  Carl  Levin 
Ranking  Minority  Member 
Committee  on  Armed  Services 
United  States  Senate 

The  Honorable  Christopher  S.  Bond 
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House  of  Representatives 
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The  NAIS  Electronic  Posting  System 


Staff  at  the  National  Aeronautics  and  Space  Administration  (nasa) 
responsible  for  developing  and  implementing  the  nasa  Acquisition  Internet 
Service  (nais)  consisted  of  a  small  group  of  procurement  and,  in  some 
cases,  technical  staff  from  each  nasa  center  and  headquarters.  They 
adopted  an  incremental  approach  to  system  development  and  operations. 
This  approach  included  the  following  steps:  (1)  general  requirements  were 
drawn  up,  (2)  commercially  available  software  options  were  researched, 
(3)  a  prototype  was  designed  and  developed  if  a  commercial  solution  was 
not  available,  (4)  a  prototype  for  testing  was  installed,  (5)  requirements 
were  refined  and  a  small  pilot  initiated,  and  (6)  the  pilot  was  expanded 
and  the  software  solution  added  to  the  nais  system. 

nasa’s  development  of  its  Electronic  Posting  System  (eps)  is  characteristic 
of  the  way  the  nais  team  applied  the  concept  of  incremental  design, 
development,  and  implementation  of  nais  features.  Prior  to  agencywide 
use  of  EPS,  each  nasa  center  had  its  own  unique  process  for  preparing  and 
posting  procurement  notices  and  solicitations  on  the  Internet.  Some 
centers  used  word  processors  while  others  used  some  form  of  automated 
document  generation  system.  The  completed  files  were  then  provided  to  a 
Web  curator  at  the  center  who  would  post  them  on  an  Internet  server  and 
manually  update  a  readable  index  of  files.  Changes  to  the  posting  process 
occurred  spontaneously  at  various  centers  with  little  central  control  or 
agreement.  The  10  centers’  differing  processes  required  10  separate  efforts 
to  implement  a  single  change  that  had  to  be  agreed  on  by  the  larger  nais 
team.  These  process  differences  also  made  it  difficult  to  maintain  a 
standard  “look  and  feel”  when  searching  out  business  opportunities.  A 
vendor  was  faced  with  10  different  entry  points  to  search. 

nais  operated  for  a  year  in  this  fashion.  Then  an  ad-hoc  group  of  nais  team 
members  began  to  study  ways  to  make  the  centers’  posting  processes 
more  uniform.  The  team  concluded  that  the  nais  posting  process  should  be 
centralized  to  a  single  server  and  that  the  application  for  posting  and 
maintaining  documents  should  be  standardized  by  using  on-line  forms  to 
be  completed  by  users,  nais  technical  team  members  then  designed  and 
developed  the  uniform  Web  posting  tools  that  were  first  piloted  and  then 
deployed  agencywide. 

According  to  nasa,  implementation  of  the  eps  eliminated  much  duplication 
among  the  centers,  standardized  posting  practices  across  all  nasa  centers, 
and  lowered  nais  maintenance  costs.  The  eps  application  provides  a 
common  look  and  feel  of  acquisition  opportunities. 
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The  NAIS  Electronic  Posting  System 


The  incremental,  evolutionary  approach  to  deploying  an  electronic  posting 
system  avoided  the  costs  and  delays  often  associated  with  attempting  a 
single  solution  to  standardize  disparate,  decentralized  systems. 
Additionally,  because  much  of  the  system  is  based  on  commercial 
standards,  commercially  available  software,  and  standard  hardware,  it  was 
easily  adopted  by  each  of  the  centers  without  major  disruptions. 

All  nais  features  have  been  built  using  commercial  Internet  standards  and 
Web  methodology  so  that  even  casual  Internet  users  can  easily  use  nais. 
For  internal  procurement  staff  using  standard  nasa  computer  hardware 
and  software,  nais  tools  may  include  user  instructions.  For  example,  the 
nais  EPS  has  an  on-line  user’s  manual  linked  directly  to  the  application.  For 
external  users  with  a  wide  variety  of  hardware  and  software,  nais  features 
are  based  on  widely  accepted  Internet  protocols  and  standards,  and  the 
features  are  designed  to  be  used  without  any  formal  training. 
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NASA’s  Business  Case  for  NAIS 


In  May  1998,  nasa  completed  and  published  an  evaluation  of  nais  in  a 
document  called  the  nais  Business  Case.  The  document  sought  to  justify 
continued  funding  for  nais  by  showing  that  the  system’s  quantitative  and 
qualitative  benefits  far  exceed  nasa’s  investment.  According  to  nasa’s 
assessment,  the  nais  tools  implemented  to  date  had  produced  outstanding 
results. 

We  found  weaknesses  in  the  evaluation’s  methodology.  For  example,  nasa 
claimed  that  the  Midrange  Pilot  initiatives,  including  nais,  reduced 
procurement  leadtimes  by  40  percent.1  We  examined  the  data  used  by  nasa 
to  support  this  calculation.  In  a  number  of  cases,  the  data  showed  that  the 
official  purchase  requests  arrived  at  nasa  procurement  offices  after 
solicitations  for  the  requested  products  or  services  had  already  been 
issued.  Consequently,  we  could  not  verify  nasa’s  claimed  40-percent 
reduction  in  leadtime.  However,  it  is  obvious  that  the  reduction  in  mail 
delays  and  in  the  manual  steps  and  efforts  needed  to  post  a  procurement 
notice  should  contribute  to  reduced  leadtimes. 

We  also  found  that  the  nais  Business  Case’s  discussion  of  the  resources 
used  in  developing  and  operating  nais  was  incomplete.  It  described  only 
the  full-time  equivalent  (fte)2  staff  years  of  effort,  both  funded  and 
unfunded,  which  were  devoted  to  nais  development  and  support  in  fiscal 
year  1997  and  did  not  reflect  the  additional  resources  such  as  technical 
support,  hardware,  software,  and  communications  services  provided  by 
various  nasa  centers. 

Because  the  evaluation  only  identified  nais  fte  staffing  for  1  year  and  did 
not  identify  hardware,  software,  and  other  costs  associated  with  nais,  we 
asked  nasa  to  provide  fiscal  year  estimates  of  the  total  costs  associated 
with  nais  development  and  ongoing  operations.  The  table  below  depicts 
nasa’s  estimates  of  the  staffing,  hardware,  software,  and  miscellaneous 
costs  associated  with  nais  for  fiscal  years  1995  through  1998.  Amounts  for 
fiscal  years  1995  through  1997  are  estimates,  while  the  amount  for  fiscal 
year  1998  is  a  projection. 


Procurement  leadtime,  broadly  speaking,  is  the  time  it  takes  to  accomplish  a  contracting  action. 
Leadtime  can  be  calculated  in  a  variety  of  ways  and  is  affected  by  many  variables.  Leadtime  in  the 
NAIS  evaluation  represented  the  period  of  time  from  receipt  of  a  purchase  request  by  the  procurement 
office  to  award  of  the  contract. 

2Federal  civilian  employment  is  generally  stated  on  an  FTE  basis.  It  is  the  total  number  of  hours 
worked  (or  to  be  worked)  divided  by  the  number  of  compensable  hours  applicable  to  each  fiscal  year. 
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Table  11.1:  Annual  NAIS  Cost  Estimates 


Fiscal  year 

NASA  staff 

Contractor 

staff 

Hardware/ 

software/misc. 

Total 

1995 

$268,620 

$56,000 

$145,000 

$469,620 

1996 

529,100 

108,500 

145,000 

$782,600 

1997 

345,090 

248,500 

185,900 

$788,490 

1998 

488,400 

280,000 

145,000 

$913,400 

User  Feedback  on  NAIS  NASA  reported  in  the  nais  Business  Case  that  data  from  the  nais  on-line 

Was  Positive  feedback  system  showed  that  over  85  percent  of  those  who  responded  to 

the  question,  “Is  nais,  as  a  whole,  beneficial?,”  answered  “yes.”  We 
reviewed  summary  reports  of  responses  tallied  from  the  nais  system  and 
found  that  of  the  2,310  persons  answering  this  question,  2,007  answered 
yes  (87  percent).  The  survey  respondents  identified  themselves  primarily 
as  small,  small  disadvantaged,  or  women-owned  businesses.  Overall,  we 
found  both  industry  and  nasa  procurement  staff  comments  about  specific 
nais  operational  features  were  positive. 

We  also  reviewed  feedback  responses  submitted  from  1995  through 
June  1998  to  an  open-ended  question  about  enhancements  that  would 
improve  nais.  As  of  June  30, 1998,  the  nais  team  had  completed  actions  or 
had  actions  underway  related  to  42  of  89  suggestions. 

Federal  officials  and  groups  representing  industry  perspectives  told  us 
that  the  nais  Business  Case  properly  identified  the  areas  in  which  both 
government  and  industry  may  experience  time  and  cost  savings  through 
an  effective  agencywide  eps.  They  noted  that  electronic  commerce 
systems  with  the  functionality  and  user-friendly  features  of  nais  have  clear 
potential  to  benefit  both  industry  and  government,  particularly  if  all  buying 
centers  are  using  the  same  system. 
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National  Aeronautics  and 
Space  Administration 

Headquarters 

Washington,  DC  20546-0001 


DEG  4  1998 

Reply  to  Attn  of:  HC 


Mr.  Allen  Li 

Associate  Director,  Defense 
Acquisition  Issues 
General  Accounting  Office 
Washington,  D.C.  20548 


Dear  Mr.  Li, 

Thank  you  for  offering  NASA  the  opportunity  to  provide 
comments  on  your  draft  report  entitled  ACQUISITION  ftgpOFM:  . 
NASA 1 s  Internet  Service  improves  Access  to  Contracting 
Information  (GAO/NS I AD-99-37) . 

As  your  staff  observed  during  the  course  of  their 
review,  we  are  very  proud  of  the  job  the  NAIS  Team  has 
done  in  pioneering  the  use  of  the  Internet  in  the  Federal 
procurement  arena.  We  are  also  pleased  that  your  review 
largely  corroborated  our  own  assessment  of  its  value. 

If  you  have  any  questions  or  require  additional 
information,  please  contact  Mr.  Tom  Deback  on  (202)  358— 
0431. 

Sincerely, 

Tom  Luedtke 

Acting  Associate  Administrator 
for  Procurement 


Page  22 


GAO/NSIAD-99-37  Acquisition  Reform 


Appendix  IV 


Comments  From  the  Office  of  Federal 
Procurement  Policy 


EXECUTIVE  OFFICE  OF  THE  PRESIDENT 
OFFICE  OF  MANAGEMENT  AND  BUDGET 
WASHINGTON,  D.  C.  20503 


January  11,  1999 


Mr.  Allen  Li 

Associate  Director 

Defense  Acquisitions  Issue 

National  Security  and  International  Affairs  Division 

U.S.  General  Accounting  Office 

Washington,  DC  20548 

Dear  Mr.  Li: 

I  appreciate  the  opportunity  to  review  your  draft  report  GAO/NS  I  AD-99-3 7,  entitled 
“Acquisition  Reform:  NASA's  Internet  Service  [NAIS]  Improves  Access  to  Contracting 
Information.”  As  a  strong  advocate  for  the  use  of  electronic  commerce  (EC)  to  improve  the 
efficiency  and  effectiveness  of  our  acquisition  processes,  I  read  your  report  on  NAIS  with  great 
interest  and  offer  the  following  comments. 

As  your  report  indicates,  NAIS  has  enabled  NASA  contracting  officers  to  easily  provide, 
and  interested  vendors  to  easily  access,  procurement  information  related  to  open  market 
purchases  above  $25,000  at  a  central  electronic  source  on  the  Internet.  NAIS  has  formed  the 
foundation  for  an  ongoing  multi-agency  pilot  of  an  electronic  posting  system  (EPS)  that,  if 
successful,  will  provide  easy  access  to  such  information  from  agencies  across  the  government 
through  a  single  point  of  entry.  For  this  reason,  I  am  encouraged  by  the  promising  findings 
identified  in  the  report  and  hope  that  current  testing  of  EPS  yields  favorable  results  as  well.  I 
anticipate  having  at  least  some  results  of  the  EPS  testing  available  in  time  for  inclusion  in  our 
next  report  to  Congress  on  EC.  In  our  report,  I  also  intend  to  describe  the  latest  plan  for,  and 
significant  agency  activities  towards,  implementing  a  government- wide  single  point  of  entry, 
including  the  feasibility  of  EPS  to  perform  this  function. 

I  appreciate  the  GAO  report's  recognition  that  current  statutory  requirements  associated 
with  publication  of  procurement  notices  and  minimum  wait  periods  -  fashioned  around  a  paper- 
based  process  ~  may  limit  the  potential  benefits  to  be  gained  by  using  EC  to  publicize  business 
opportunities  and  related  information.  I  intend  to  carefully  study  what  type  of  legislative  action 
might  be  feasible. 

Not  long  ago,  the  GAO  challenged  the  Administration  to  work  with  Congress  in  finding 
better  ways  to  bring  about  improvements  in  the  procurement  process  through  the  use  of  EC.  This 
challenge  served  as  a  catalyst  for  a  bipartisan  effort  that  resulted  in  a  new  legislative  framework. 
This  framework,  which  expands  agencies'  ability  to  use  EC  more  effectively,  is  revitalizing 


OFFICE  OF  FEDERAL 
PROCUREMENT  POLICY 
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agency  interest  in  EC  and  enabling  better  migration  of  agency  resources  to  EC  applications  that 
hold  the  greatest  promise  of  process  improvement.  I  am  hopeful  that  the  positive  findings  and 
recommendations  of  this  report  will  pave  the  way  for  additional  productive  dialogue  and  uses  of 
EC  that  further  enhance  access  to  contracting  opportunities. 


Sincerely? 


Deidre  A.  Lee 
Administrator 
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Major  Contributors  to  This  Report 


National  Security  and 
International  Affairs 
Division,  Washington, 
D.C. 


Delores  Cohen 
Richard  J.  Herley 
Dan  Hoagland 
Stefano  Petrucci 
Patricia  D.  Slocum 


Atlanta  Field  Office 


Bobby  Hall 


Office  of  General 
Counsel,  Washington, 
D.C. 


John  Carter 


(707346) 
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